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I.  INTRODUCTIONS 


One  of  the  outcomes  of  the  19*  meeting  of  The  Technical  Cooperation  Program  (TTCP) 

Sub  Group  K  Technical  Panel  KTP-2  was  a  set  of  track  metrics  [1].  The  purpose  of  these  metrics 
was  to  enable  a  qualitative  comparison  of  the  effectiveness  of  different  radar  tracking  algorithms 
in  the  face  of  various  target  situations.  The  complex  and  highly  variable  target  situations  that 
radars  face  have  long  made  the  development  of  a  quantitative  set  of  metrics  that  yield  absolute 
scoring  virtually  impossible.  These  metrics  now  provide  the  radar  engineer  with  a  consistent 
basis  upon  which  to  compare  the  performance  of  various  tracking  algorithms.  This  report  will 
detail  the  results  of  applying  these  metrics  to  data  obtained  during  a  Multi-Role  Survivable  Radar 
(MRSR)  tracking  test  and  presented  at  the  2C'  meeting  of  the  KTP-2  group. 

11.  TEST  DESCRIPTION 

The  MRSR  is  an  advanced  development  track-while-scan  air  defense  radar  developed  by  the 
U.  S.  Army  Aviation  and  Missile  Command  (AMCOM).  Near  hemispheric  search  and  track 
coverage  are  achieved  using  360-degree  mechanical  rotation  with  phase  steered  elevation 
coverage.  Tracking  in  the  MRSR  is  handled  by  the  Track  Processor  which  is  a  self-contained 
software  module  within  MRSR’s  Radar  Data  Processor  (RDP).  The  inputs  to  the  Track  Processor 
are  detection  reports  which  come  from  the  Dwell  Management  Processor,  another  self-contained 
software  module  within  the  RDP.  The  Track  Processor  is  responsible  for  all  track  initiation  and 
maintenance  functions.  Next  scan  location  prediction  and  measured  data  smoothing  are 
performed  using  an  a-P  track  filter.  When  a  scan  fails  to  yield  a  detection  with  which  to  update  a 
track,  track  coasting  is  performed  using  the  previous  prediction. 

The  data  analyzed  for  this  report  were  collected  during  an  August  1994  tracking  test.  The 
target  aircraft  for  this  test  was  an  F-86  equipped  with  a  differential  Global  Positioning  System 
(GPS)  pod.  This  pod  provided  time-space  truth  data  with  3m  accuracy  in  the  x,  y,  and  z 
dimensions.  The  runs  analyzed  were  all  3-g  S-weaves  flown  radially  with  respect  to  the  MRSR. 

A  run  is  comprised  of  an  outbound  and  inbound  leg.  Figures  1  through  3  show  the  GPS  track 
data  for  each  run  and  the  corresponding  MRSR  track  data. 

An  examination  of  the  MRSR  tracks  for  each  of  the  three  runs  shows  some  notable  and 
interesting  differences  between  them.  For  Run  1,  the  tracking  results  are  almost  ideal  with  a 
single  track  covering  essentially  the  entire  run.  Run  2  provides  an  excellent  example  of  the 
bifurcation  that  often  occurs  in  the  MRSR  tracker  (tracks  88  and  76)  as  well  as  a  single  long 
track.  For  MRSR,  bifurcation  occurs  when  multiple  detections  on  the  same  target  fail  to  collapse 
to  a  single  detection  before  entering  the  tracker.  This  causes  the  tracker  to  setup  multiple  tracks 
on  the  same  target.  Track  segmenting  is  the  distinguishing  feature  of  Run  3.  The  MRSR  data  is 
composed  of  multiple  segments  covering  less  than  50  percent  of  the  run.  Due  to  the  uniqueness 
of  each  run,  the  metrics  were  applied  to  each  separately  so  that  the  effects  of  each  tracking 
phenomena  can  be  better  understood. 
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III.  TRACK  METRIC  ANALYSIS 


The  KTP-2  track  metrics  are  divided  into  two  groups  depending  on  whether  the  track  is  real 
or  false.  Only  the  real  track  metrics  were  used  for  this  analysis  since  no  false  tracks  exist  within 
the  data  set.  For  this  analysis,  the  definition  for  each  metric  from  Reference  [1]  will  be  given 
along  with  how  the  metric  was  applied  to  the  MRSR  data  and  the  results  obtained. 

A.  Track  Initiation  Delay 

Definition:  The  length  of  time  after  the  initial  detection  that  it  takes  to  report  or  display 

the  track. 


Implementation:  The  elapsed  time  from  the  first  verifiable  signal  processor  detection 
until  a  track  on  the  target  is  reported. 

Results:  Due  to  a  data  recording  glitch,  no  initiation  data  was  recorded  for  Track  81 
Run  1,  Tracks  120  and  88  Run  2,  and  Track  73  Run  3.  Tracks  48  and  76  Run  2  were  range 
resolved  and  displayed  on  the  first  scan  during  which  they  were  detected.  Tracks  54  and  16  Run 
3  were  range  resolved  and  displayed  one  scan  after  their  initial  detection  which  results  in  a  2- 
second  delay.  For  this  limited  data  set,  track  initiation  delay  can  be  deemed  insignificant. 

B.  Track  Life 

Definition:  The  percentage  of  time  the  track  is  displayed  or  reported  regardless  of  the 
number  of  track  breaks  or  track  number  changes. 

Implementation:  The  total  time  the  target  is  displayed  as  a  percentage  of  the  elapsed 
time  from  the  first  track  report  on  the  target  to  the  last  track  report. 

Results:  As  can  be  seen  from  Figures  1  and  2,  Runs  1  and  2  contain  no  intervals 
between  the  first  and  last  track  points  without  a  displayed  track.  Consequently,  their  track  life  is 
100  percent.  Figure  3  shows  two  large  track  breaks  over  the  course  of  Run  3  resulting  in  a  track 
life  of  77.6  percent. 

C.  Number  of  Track  Numbers  Associated  with  the  Track 

Definition:  Trackers  which  drop  and  restart  tracks  without  linking  the  segments  (and 
do  not  maintain  the  same  track  number)  should  be  heavily  penalized. 

Implementation:  The  total  number  of  track  numbers  correlating  to  the  target  of  interest 
for  each  run. 

Results:  For  Run  1,  MRSR  maintained  a  single  continuous  track  throughout  the  run  so 
only  one  track  number  was  used.  During  Run  2,  the  MRSR  tracker  experienced  some  light  track 
segmentation  as  well  as  bifurcation  resulting  in  the  use  of  four  track  numbers.  However,  it  should 
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be  noted  that  two  of  the  numbers  are  coincident  with  other  track  numbers  as  a  result  of  the 
bifurcation.  Run  3  is  heavily  segmented  and  consequently  used  three  unique  track  numbers.  The 
MRSR  tracker  contains  no  logic  to  attempt  to  perform  track  linkage,  so  by  default,  when  track 
segmentation  occurs  it  will  perform  poorly  against  this  metric. 

D.  Number  of  Track  Breaks  with  Constant  Track  Number 

Definition:  Trackers  which  contain  breaks  in  a  track  but  maintain  track  number 
continuity  are  penalized  relative  to  one  which  has  no  breaks  but  is  superior  to  one  with  breaks 
and  changes  in  the  track  number. 

Implementation:  This  metric  is  not  applicable  to  the  MRSR  tracker  since  it  makes  no 
attempt  to  perform  track  linkage.  Track  segments  with  the  same  track  number  are  a  purely 
random  event  in  the  MRSR  tracker. 

E.  Total  Duration  of  Track  Divergence 

Definition:  The  percentage  of  total  track  life  that  the  tracker  output  position  exceeds 
the  actual  track  position  by  some  specified  multiple  of  sensor  resolution  cells. 

Implementation:  The  percentage  of  total  track  life  that  the  tracker  output  position 
exceeds  the  actual  track  position  by  two  standard  deviations  of  MRSR’ s  typical  track  accuracy 
(Section  III.G.). 

Results:  As  would  be  expected  for  a  long  track.  Track  81  Run  1  has  a  relatively  low 
divergence,  only  21.2  percent,  third  lowest  in  the  data  set.  Tracks  120,  88  and  76  Run  2  had 
divergences  of  46.2  percent,  50.0  percent,  and  100  percent,  respectively.  These  results  are  not 
surprising  given  the  short  duration  of  these  tracks  and  the  fact  that  Tracks  88  and  76  are 
bifurcated  tracks.  Track  48  Run  2  has  the  lowest  divergence  of  any  track,  13.3  percent.  This  is 
consistent  with  its  also  being  one  of  the  longest  tracks  in  the  data  set.  Tracks  73  and  54  Run  3 
have  divergences  of  20.0  percent  and  42.9  percent,  respectively.  Track  54’ s  divergence  is 
consistent  with  the  fact  that  it  is  a  short  track  in  the  middle  of  a  long  turn.  Track  73’s  divergence 
is  the  second  lowest  of  the  data  set  which  is  unexpected  since  it  is  a  short  track  early  in  a 
maneuver.  Track  16  Run  3’s  divergence  of  36.4  percent  is  somewhat  surprising  given  that  this  is 
a  medium  length  track.  Overall,  the  divergences  are  consistent  with  MRSR’s  intended  role  of 
providing  cueing  for  a  mid-course  guided  missile  system. 

F.  Number  of  Divergent  Tracks 

Definition:  The  number  of  times  that  the  tracker  output  position  exceeds  the  actual 
track  position  by  some  specified  multiple  of  sensor  resolution  cells. 

Implementation:  The  number  of  reports  that  the  tracker  output  position  exceeds  the 
actual  position  by  2  standard  deviations  of  MRSR’s  typical  track  accuracy  (Section  ni.G.). 
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Results:  This  metric  is  simply  the  numeric  values  used  to  compute  the  Track 
Divergence  (Section  in.E.).  For  Track  81  Run  1,  this  was  29  reports  out  of  a  total  of  137.  For 
Tracks  120,  88,  48,  and  76  Run  2;  it  was  6  of  13,  4  of  8,  16  of  120,  and  6  of  6,  respectively.  For 
Track  73,  54  and  16  Run  3;  it  was  3  of  15,  3  of  7  and  20  of  55,  respectively. 

G.  Track  Accuracy 

Definition:  The  accuracy  of  the  tracker  output.  This  can  only  be  determined  for  those 
cases  where  ground  truth  data  is  available. 

Implementation:  Global  statistics  for  range,  azimuth,  and  elevation  errors  were 
calculated  using  the  longest  track  from  each  run. 

Results:  The  global  mean  errors  for  range,  azimuth,  and  elevation  were  -28.2  m,  -0.20, 
and  -0.31  degrees,  respectively.  These  errors  are  typically  caused  by  bias  errors  between  the 
origins  of  the  radar  coordinate  system  and  the  truth  data  coordinate  system.  The  global  standard 
deviations  for  range,  azimuth,  and  elevation  were  38.7  m,  0.27,  and  0.59  degrees,  respectively. 
These  values  represent  the  radar’s  typical  track  accuracy  as  mentioned  in  Sections  III.F.  and  F. 

H.  Number  of  Track  Swaps 

Definition:  Crossing  targets  can  result  in  swapping  of  track  numbers;  tracking  systems 
which  have  difficulties  with  these  types  of  targets  should  be  penalized. 

Implementation:  This  test  involved  only  one  aircraft  so  no  evaluation  of  this  metric  is 
possible  for  this  data  set. 

I.  Number  and  Percentage  of  Omitted  Tracks 

Definition:  The  total  number  of  tracks  which  are  completely  missed  by  a  tracker  and 
that  number  as  percentage  of  the  total  number  of  valid  tracks  in  the  data  set. 

Implementation:  Given  that  this  test  involved  only  one  aircraft  flying  profiles  radially 
to  the  radar,  this  metric  could  not  be  properly  evaluated. 
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IV.  CONCLUSION 


This  report  has  presented  the  results  of  an  analysis  based  on  the  KTP-2  Group’s  Real  Track 
Metrics  of  MRSR’s  tracking  performance  against  a  maneuvering  target.  Overall,  the  MRSR’s 
tracker  performed  well  against  a  difficult  target  scenario.  Over  the  entire  test,  the  target  was 
under  track  92.5  percent  of  the  time  with  sufficient  track  accuracy  to  support  target  engagement 
with  a  mid-course  guided  missile.  The  principle  problem  noticed  during  the  analysis  was  track 
bifurcation  for  near-in  targets.  This  problem  is  caused  by  a  lack  of  track-to-track  correlation  logic 
within  the  MRSR  tracker. 
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Figure  1.  Truth  Data  and  MRSR  Track  Data  for  Run 
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Figure  2.  Truth  Data  and  MRSR  Track  Data  for  Run  2 
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